chore(deps): refresh toolchains and scanner runtime - #51
Merged
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. |
clawsweeper
Bot
added
merge-risk: 🚨 compatibility
|
Codex review: needs real behavior proof before merge. Reviewed August 31, 2026, 5:19 AM ET / 09:19 UTC. ClawSweeper reviewWhat this changesUpdates Go and npm baselines, fixes npm 12 package tarball parsing, refreshes six sandbox runtime tool pins, and counts Snyk 0.6 named risk indicators in CLI summaries. Merge readiness⛔ Blocked until stronger real behavior proof is added - 4 items remain Keep open pending the repository’s documented runtime-upgrade evidence: the npm 12 packaging repair and Snyk summary change are focused and tested, but the default sandbox scanner updates still lack candidate-versus-current benchmark artifacts. Priority: P2 Review scores
Verification
How this fits togetherClawScan runs scanners in a default Docker sandbox, preserves their raw JSON, and reports a concise CLI summary. Its npm packaging script builds and smoke-tests distributable CLI binaries, while the runtime image supplies command-backed scanner tools. flowchart LR
A[Operators and CI] --> B[ClawScan CLI]
B --> C[Default Docker sandbox]
C --> D[Scanner tools]
D --> E[Raw scanner JSON]
E --> F[CLI issue summary]
B --> G[npm package build]
G --> H[Installed CLI smoke test]
Before merge
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Land the backward-compatible npm parser and Snyk summary only after a candidate sandbox image has documented benchmark comparisons against the current runtime and any verdict or failure-rate drift is explained. Do we have a high-confidence way to reproduce the issue? Yes—source inspection shows current main only accepts an array from npm pack JSON, while the supplied npm 12 object shape reaches its error path. The PR body also provides a real npm 12 after-fix packaging trace, though this read-only review did not execute it. Is this the best way to solve the issue? Yes for the npm 12 compatibility and Snyk summary fixes; retaining both old and new npm shapes is narrow and maintainable. The runtime pin portion still needs the repository-prescribed comparative evidence before it is ready to merge. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 9218ebfaff8c. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
|
|
ClawSweeper status: review started. I am starting a fresh review of this pull request: chore(deps): refresh toolchains and scanner runtime This is item 1/1 in the current shard. Shard 0/1. This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking. Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update the Go toolchain to 1.27.0 and npm to 12.0.2. npm 12 returns
npm pack --jsonas an object keyed by package name, which broke package staging. Parse that documented format while retaining the array format used by supported older npm versions, cover both with regression tests, and run the existing CI packaging smoke on npm 12.Runtime dependency refresh, generated by
node scripts/update-runtime-tool-versions.mjs:8f37cfa1539c1150Go library requirements are current after
go get -u ./...andgo mod tidy; the npm wrapper has no package dependencies. Gradio 6.26.0, Hugging Face Hub 1.29.0, AIG 0.2.1, AgentVerus 0.8.1, and all 18 external Actions dependencies are already current. The Go requirement now matches the latest stable toolchain and the contributor documentation. No new library dependencies or lockfiles are introduced.Migration references: npm 12 release notes, Go 1.27 release notes.
Codex review identified that the CLI summary did not count Snyk 0.6 named risk indicators. Count
risk_indexesmaps without normalizing or discarding raw evidence, add a representative fixture-backed CLI regression, and document the upstream schema migration for operator-owned gate rules. The adapter already preserves arbitrary valid upstream JSON and its CLI flags remain supported by Snyk 0.6.Local validation:
go test -count=1 ./...passed every package;go build ./...,go vet ./...,go mod verify,actionlint, andgit diff --checkpassed.node --test npm/clawscan/test/*.test.mjs scripts/*.test.mjspassed all 27 tests.make docs-sitebuilt seven pages, andmake release VERSION=v0.0.0-testbuilt all six local archives.Built CLI live proof:
The same built CLI completed a live SkillTrustBench fetch/materialization/static scan:
Packaging proof, including six-platform build, pack, install, version check, and a real scan:
A separately installed npm 12 tarball reported
clawscan v0.0.0and completed a synthetic benign-skill scan with one completed scanner and zero errors. Black-box input variation producedstatic.pipe_to_shellfor a remote-script-to-shell instruction; an unknown scanner exited 1 withUnknown scanner: nonexistent-scanner.The repository has no existing changelog; the user-visible Snyk output migration is documented in
docs/scanners.md. The unchanged Python requirements resolved successfully. The Gradio app served/configover localhost with HTTP 200 and 12 components, loaded two fixture rows, and validated the 3,368-case upload fixture without publishing.CI context: default-branch build/test CI is green. The older failed Security Signals publisher is an operational publication workflow, now manual-only, not a current build/test failure. Local reproduction still reports
metadata benchmark.dataset must be clawhub-security-signals; the validator requires a recorded revision but fetches the current case set without using it. Historical-data validation needs a separate revision-aware fix before rerunning publication. No assertions, tests, or jobs were weakened, and no production dataset was changed.Snyk 0.6 fixture proof through the rebuilt CLI (synthetic upstream-shaped evidence, no authenticated API claim):
Codex autoreview passed after the Snyk summary correction:
autoreview clean: no accepted/actionable findings reported. The review verified the unchanged raw-JSON adapter alongside the diff.The candidate Docker image built locally and all eight installed CLIs passed help checks with networking disabled. Before merging, retain the existing runtime upgrade gate: compare judge-backed SkillTrustBench and Security Signals results for the operator's configured profile. This PR does not claim an authenticated judge-quality comparison. The existing runtime's two-case Security Signals probe preserved evidence but returned no prediction verdict; Cisco 2.0.12 also rejected both samples because their
SKILL.mdfrontmatter lacked the requirednamefield.Runtime live proof:
docker build -t clawscan-runtime:deps-refresh-20260830 docker/clawscan-runtime docker run --rm --network none clawscan-runtime:deps-refresh-20260830 sh -ec 'for tool in codex claude aig-skill-scan skillspector snyk-agent-scan socket agentverus-scanner skill-scanner; do "$tool" --help >/dev/null; printf "%s: help OK\n" "$tool"; done'The help checks emitted expected missing-credential and disabled-network warnings; no credentials were supplied. Initial Docker startup was slow on the shared host, including a timed-out 60-second minimal launch probe; the eight-tool smoke subsequently completed successfully.
The identical command with
--sandbox-image ghcr.io/openclaw/clawscan-runtime:latestandbenchmark-baseline.jsonproduced four completed scans, zero failures, and two abstentions. These are scanner-evidence comparisons; no judge was configured, so aggregate accuracy is not a quality measurement.case_04866(malicious)case_01984(clean)reference_unresolved98dca0ef…manifest_parse_error42d41782…The Security Signals candidate command was:
Both SkillSpector scans completed and their artifact was preserved, but the command exited 1 with
case 98dca0efb3f965c2246b192320cb9526e55a850d23c4ca95e769c97e8ca5bb66 has no prediction verdict. This is not a passing judged benchmark. Before merge, review the new incomplete-coverage recommendations and run the configured judge-backed runtime comparison. Also migrate any operator-owned Snyk gate paths that depend on the 0.5 JSON schema.PR build/test CI, CodeQL, and the multi-platform Runtime Image build passed. Dependency review and submission validation also passed. The PR remains unmerged.